2 research outputs found
NeuralClothSim: Neural Deformation Fields Meet the Kirchhoff-Love Thin Shell Theory
Cloth simulation is an extensively studied problem, with a plethora of
solutions available in computer graphics literature. Existing cloth simulators
produce realistic cloth deformations that obey different types of boundary
conditions. Nevertheless, their operational principle remains limited in
several ways: They operate on explicit surface representations with a fixed
spatial resolution, perform a series of discretised updates (which bounds their
temporal resolution), and require comparably large amounts of storage.
Moreover, back-propagating gradients through the existing solvers is often not
straightforward, which poses additional challenges when integrating them into
modern neural architectures. In response to the limitations mentioned above,
this paper takes a fundamentally different perspective on physically-plausible
cloth simulation and re-thinks this long-standing problem: We propose
NeuralClothSim, i.e., a new cloth simulation approach using thin shells, in
which surface evolution is encoded in neural network weights. Our
memory-efficient and differentiable solver operates on a new continuous
coordinate-based representation of dynamic surfaces, i.e., neural deformation
fields (NDFs); it supervises NDF evolution with the rules of the non-linear
Kirchhoff-Love shell theory. NDFs are adaptive in the sense that they 1)
allocate their capacity to the deformation details as the latter arise during
the cloth evolution and 2) allow surface state queries at arbitrary spatial and
temporal resolutions without retraining. We show how to train our
NeuralClothSim solver while imposing hard boundary conditions and demonstrate
multiple applications, such as material interpolation and simulation editing.
The experimental results highlight the effectiveness of our formulation and its
potential impact.Comment: 27 pages, 22 figures and 3 tables; project page:
https://4dqv.mpi-inf.mpg.de/NeuralClothSim
State of the Art in Dense Monocular Non-Rigid 3D Reconstruction
3D reconstruction of deformable (or non-rigid) scenes from a set of monocular
2D image observations is a long-standing and actively researched area of
computer vision and graphics. It is an ill-posed inverse problem,
since--without additional prior assumptions--it permits infinitely many
solutions leading to accurate projection to the input 2D images. Non-rigid
reconstruction is a foundational building block for downstream applications
like robotics, AR/VR, or visual content creation. The key advantage of using
monocular cameras is their omnipresence and availability to the end users as
well as their ease of use compared to more sophisticated camera set-ups such as
stereo or multi-view systems. This survey focuses on state-of-the-art methods
for dense non-rigid 3D reconstruction of various deformable objects and
composite scenes from monocular videos or sets of monocular views. It reviews
the fundamentals of 3D reconstruction and deformation modeling from 2D image
observations. We then start from general methods--that handle arbitrary scenes
and make only a few prior assumptions--and proceed towards techniques making
stronger assumptions about the observed objects and types of deformations (e.g.
human faces, bodies, hands, and animals). A significant part of this STAR is
also devoted to classification and a high-level comparison of the methods, as
well as an overview of the datasets for training and evaluation of the
discussed techniques. We conclude by discussing open challenges in the field
and the social aspects associated with the usage of the reviewed methods.Comment: 25 page